Mobile terminal and control method thereof

ABSTRACT

A method includes displaying an image on a display of a mobile terminal, wherein the image depicts an object; displaying information simultaneously with the image, wherein the information relates to the object at a particular time point of any of a plurality of different time points of a timeline; and changing the displayed information to reflect a change in the particular time point to another time point of the plurality of different time points of the timeline, wherein the changed displayed information is displayed simultaneously with the image and relates to the object at the another time point.

CROSS-REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. §119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Patent ApplicationNo. 10-2010-0055941, filed on Jun. 14, 2010, the contents of which ishereby incorporated by reference herein in its entirety.

FIELD OF THE INVENTION

The present disclosure relates to a mobile terminal, and moreparticularly, to a mobile terminal and a control method of displayinginformation.

DISCUSSION OF THE RELATED ART

As functions of terminals such as personal computers, laptop computers,cellular phones diversify, the terminals become multimedia playershaving multiple functions for capturing pictures or moving images,playing music, moving image files and games and receiving broadcastingprograms.

Terminals can be categorized as mobile terminals and stationaryterminals. The mobile terminals can be further comprised of handheldterminals and vehicle mount terminals according to whether users canpersonally carry the terminals. Conventional terminals including mobileterminals provide an increasing number of complex and various functions.

To support and enhance the increasing number of functions in a terminal,improving a structural part and/or a software part of the terminal wouldbe desirable.

SUMMARY

In a first embodiment of the present invention, a method includesdisplaying an image on a display of a mobile terminal, wherein the imagedepicts an object; displaying information simultaneously with the image,wherein the information relates to the object at a particular time pointof any of a plurality of different time points of a timeline; andchanging the displayed information to reflect a change in the particulartime point to another time point of the plurality of different timepoints of the timeline, wherein the changed displayed information isdisplayed simultaneously with the image and relates to the object at theanother time point.

In a second embodiment of the present invention, a method of controllinginformation in a mobile terminal includes displaying an image on adisplay of a mobile terminal, wherein the image depicts an object;displaying information simultaneously with the image, wherein theinformation relates to the object at a particular time point of any of aplurality of different time points of a timeline; and changing thedepicted object to a different version to reflect a change in theparticular time point to another time point of the plurality ofdifferent time points of the timeline, wherein the different version ofthe depicted object generally corresponds to a state of the object atthe another time point, and wherein the changed depicted object isdisplayed simultaneously with the information.

In a third embodiment of the present invention, a method of controllinginformation in a mobile terminal includes displaying an image on adisplay of a mobile terminal, wherein the image depicts an object;displaying information simultaneously with the image, wherein theinformation relates to the object at a particular time point of any of aplurality of different time points of a timeline; changing the displayedinformation to reflect a change in the particular time point to anothertime point of the plurality of different time points of the timeline;and changing the depicted object to reflect the object at the anothertime point, wherein the changed displayed information is displayedsimultaneously with the changed depicted object.

In a fourth embodiment of the present invention, a mobile terminalincludes a display; and a controller configured to display an image onthe display, wherein the image depicts an object; display informationsimultaneously with the image, wherein the information relates to theobject at a particular time point of any of a plurality of differenttime points of a timeline; and change the displayed information toreflect a change in the particular time point to another time point ofthe plurality of different time points of the timeline, wherein thechanged displayed information is displayed simultaneously with the imageand relates to the object at the another time point.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention.

FIG. 1 is a block diagram of a mobile terminal according to anembodiment of the present invention.

FIG. 2 is a conceptual diagram for explaining a proximity depth of aproximity sensor according to an embodiment of the present invention.

FIG. 3A is a front perspective view of the mobile terminal according toan embodiment of the present invention.

FIG. 3B is a rear perspective view of the mobile terminal according toan embodiment of the present invention.

FIGS. 3C and 3D illustrate versions of the mobile terminal 100 accordingto an embodiment of the present invention.

FIG. 4 is a conceptual view illustrating a configuration of a CDMAwireless communication system communicating with the mobile terminalaccording to an embodiment of the present invention.

FIG. 5 is a flowchart showing an operation of the mobile terminalaccording to an embodiment of the present invention.

FIGS. 6A and 6B are views describing the operation of the mobileterminal according to an embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation of capturing an imageaccording to an embodiment of the present invention.

FIGS. 8A and 8B are views for explaining the operation of the mobileterminal according to an embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation of acquiring geographicinformation according to an embodiment of the present invention.

FIG. 10 is a visual representation of acquired geographic informationfor operating the mobile terminal according to an embodiment of thepresent invention.

FIG. 11 is a flowchart illustrating an operation of acquiringinformation about an object included in a captured image according to anembodiment of the present invention.

FIG. 12 is a flowchart illustrating the operation of the mobile terminalaccording to an embodiment of the present invention.

FIG. 13 is a flowchart illustrating an operation of setting a specifictime according to an embodiment of the present invention.

FIGS. 14 through 19 are timelines displayed via a display module of themobile terminal according to an embodiment of the present invention.

FIG. 20 is a flowchart illustrating an operation of displaying objectinformation at a set time according to an embodiment of the presentinvention.

FIGS. 21 through 23 illustrate operation of displaying objectinformation at a set time via the display module of the mobile terminalaccording to an embodiment of the present invention.

FIGS. 24A through 27B are views for explaining an operation of themobile terminal according to an embodiment of the present invention.

DETAILED DESCRIPTION

The above and other aspects, features and advantages of the presentdisclosure will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings. Hereinafter, animplementation of this document will be described in detail withreference to the attached drawings. Like reference numerals refer tolike elements throughout the specification. In describing the presentdisclosure, detailed descriptions of well-known functions orconfigurations will be omitted in order to not necessarily obscure thesubject matter of the present disclosure.

The suffixes ‘module’, ‘unit’ and ‘part’ may be used for elements inorder to facilitate the disclosure herein. Significant meanings or rolesmay not be given to the suffixes themselves and it is understood thatthe ‘module’, ‘unit’ and ‘part’ may be used together or interchangeably.

Referring to FIG. 1, the mobile terminal 100 may refer to an electronicdevice among a plurality of external electronic devices on a networkaccording to an embodiment of the present invention, which will bedescribed in more detail with reference to the drawings. The mobileterminal 100 can include at least a cellular phone, a smart phone, alaptop computer, a digital broadcasting terminal, personal digitalassistants (PDA), a portable multimedia player (PMP), or a navigationsystem.

The mobile terminal 100 according to an embodiment of the presentinvention includes a wireless communication unit 110, an A/V(audio/video) input unit 120, a user input unit 130, a sensing unit 140,an output unit 150, a memory unit 160, an interface unit 170, acontroller unit 180, and a power supply unit 190. FIG. 1 shows themobile terminal 100 having various components, but it is understood thatimplementing all of the illustrated components is not required. Greateror fewer components may alternatively be implemented.

The wireless communication unit 110 can include one or more componentsthat permit wireless communication between the mobile terminal 100 and awireless communication system or network within which the mobileterminal 100 is located. For example, the wireless communication unit110 can include a broadcast receiving module 111, a mobile communicationmodule 112, a wireless Internet module 113, a short-range communicationmodule 114, and a position-location module 115.

The broadcast receiving module 111 receives a broadcast signal and/orbroadcast associated information from an external broadcast managingserver (not shown) via a broadcast channel. The broadcast channel mayinclude a satellite channel and a terrestrial channel.

The broadcast managing server generally refers to a server whichgenerates and transmits the generated broadcast signal and/or broadcastassociated information or a server which is provided with a previouslygenerated broadcast signal and/or broadcast associated information andthen transmits the provided broadcast signal or information to themobile terminal 100. The transmitted broadcast signal may be implementedas a television (TV) broadcast signal, a radio broadcast signal, and adata broadcast signal. The transmitted broadcast signal may be combinedwith a TV or radio broadcast signal.

The broadcast associated information can include information associatedwith a broadcast channel, a broadcast program, and a broadcast serviceprovider. The broadcast associated information can be provided via amobile communication network, and be received by the mobilecommunication module 112 via a broadcast signal antenna 116.

The broadcast associated information can be implemented in variousforms. For example, broadcast associated information may include anelectronic program guide (EPG) related to digital multimediabroadcasting (DMB) and electronic service guide (ESG) related to digitalvideo broadcast-handheld (DVB-H).

The broadcast receiving module 111 may be configured to receivebroadcast signals transmitted from various types of broadcastingsystems. The broadcasting systems can include digital multimediabroadcasting-terrestrial (DMB-T), digital multimediabroadcasting-satellite (DMB-S), digital video broadcast-handheld(DVB-H), media forward link only (MediaFLO®) and integrated servicesdigital broadcast-terrestrial (ISDB-T). Greater or fewer broadcastingsystems can be received by the broadcast receiving module 111.

The broadcast signal and/or broadcast associated information received bythe broadcast receiving module 111 may be stored in a storage device,such as the memory unit 160.

The mobile communication module 112 transmits and receives wirelesssignals between one or more network entities (e.g., base station,external terminal, and server) via the broadcast signal antenna 116. Thetransmitted and received wireless signals may represent audio, video,and data signals according to text or multimedia message transmissions.

The wireless Internet module 113 supports Internet access for the mobileterminal 100. The wireless Internet module 113 may be internally orexternally coupled to the mobile terminal 100. In this case, thewireless Internet technology supported by the wireless Internet module113 can include Wireless LAN (WLAN), Wireless Fidelity (Wi-Fi™), Wibro(Wireless broadband), World Interoperability for Microwave Access(WiMAX), and High Speed Downlink Packet Access (HSDPA).

The short-range communication module 114 facilitates relativelyshort-range communications. Suitable technologies for implementing theshort-range communication module 114 can include radio frequencyidentification (RFID), infrared data association (IrDA), ultra-wideband(UWB), Bluetooth® and ZigBee®.

The position information module 115 is a module for identifying orotherwise obtaining a position of the mobile terminal. The positioninformation module 115 can acquire position information using a globalnavigation satellite system (GNSS). Here, GNSS refers to radionavigation satellite systems that orbit the earth and transmit referencesignals so that the location of certain types of radio navigationreceivers on the earth's surface can be determined or approximated. GNSSincludes a global positioning system (GPS) managed by the USA, Galileomanaged by Europe, global orbiting navigational satellite system(GLONASS) managed by Russia, COMPASS managed by China, and quasi-zenithsatellite system (QZSS) managed by Japan.

As one representative example of GNSS, the position information module115 may be a GPS (Global Positioning System) module. The GPS module 115can calculate information on distances between one point (object) and atleast three satellites, information on the time when the distanceinformation is measured, and use the obtained distance information totriangulate three-dimensional position information on the point (object)according to latitude, longitude and altitude at a predetermined time.Moreover, a method of calculating position and time information usingthree satellites and correcting the calculated position and timeinformation using another satellite can also be used. In addition, theGPS module 115 continuously calculates the current position in real timeand calculates velocity information using the position information.

The audio/video (A/V) input unit 120 can be configured to provide audioor video signal input to the mobile terminal 100. The A/V input unit 120can include a camera module 121, a microphone module 122, a flash module123 and a mirror module 124. The camera module 121 can receive andprocess image frames of still pictures (e.g., photographs) obtained byan image sensor when the mobile terminal 100 is in a photographing mode,and alternatively, receive and process moving picture data (e.g., video)when the mobile terminal 100 is in a video call mode. The processedimage frames can be displayed by the output unit 150, such as a displaymodule 151.

The image frames processed by the camera module 121 can be stored in thememory unit 160 or can be externally transmitted via the wirelesscommunication unit 110. At least two camera modules 121 can be providedin the mobile terminal 100 depending on the usage environment.

The microphone module 122 receives an external audio signal while themobile terminal 100 is in a particular mode, such as a phone call mode,a recording mode and a voice recognition mode. The external audio signalis processed and converted into digital audio data. The digital audiodata is transformed into a format transmittable to a mobilecommunication base station via the mobile communication module 112 whenthe mobile terminal 100 is in a call mode. The microphone module 122 caninclude assorted noise removing algorithms to remove noise generatedwhen receiving the external audio signal. The flash module 123 canprovide lighting in conjunction with the camera module 121 obtainingimages of the external environment. The mirror module 124 can provide auser with a reflective surface.

The user input unit 130 generates input data responsive to usermanipulation of one or more associated input devices. Examples of suchinput devices can include a keypad, a dome switch, a touchpad (e.g.,static pressure/capacitance), a jog wheel, and a jog switch.

The sensing unit 140 provides sensing signals for controlling operationsof the mobile terminal 100 using status measurements of various aspectsof the mobile terminal 100. For example, the sensing unit 140 may detectan open/close status of the mobile terminal 100, a relative positioningof components (e.g., a display and keypad) of the mobile terminal 100, achange of position of the mobile terminal 100 or a component of themobile terminal 100, a presence or absence of user contact with themobile terminal 100, or an orientation or acceleration/deceleration ofthe mobile terminal 100.

In another example, when the mobile terminal 100 is configured as aslide-type mobile terminal, the sensing unit 140 may sense whether asliding portion of the mobile terminal 100 is open or closed. In anotherexample, the sensing unit 140 can sense the presence or absence of powerprovided by the power supply unit 190, the presence or absence of acoupling or connection between the interface unit 170 and a deviceexternal to the mobile terminal 100. The sensing unit 140 can include aproximity sensor 141.

The output unit 150 generates outputs relevant to senses of sight,hearing, and touch. The output unit 150 can include the display module151, an audio output module 152, an alarm module 153, a haptic module154 and an earphone module 156.

The display module 151 can be implemented to visually display or outputinformation associated with the mobile terminal 100. For example, if themobile terminal 100 is operating in a phone call mode, the displaymodule 151 can provide a user interface (UI) or graphical user interface(GUI) which includes information associated with placing, conducting,and terminating a phone call. In another example, if the mobile terminal100 is in a video call mode or a photographing mode, the display module151 may additionally or alternatively display images which areassociated with the photographing or video call modes, the UI or theGUI.

The display module 151 may be implemented using one or more displaytechnologies which include a liquid crystal display (LCD), a thin filmtransistor-liquid crystal display (TFT-LCD), an organic light-emittingdiode display (OLED), a flexible display and a three-dimensional (3D)display. Moreover, a transparent display can be implemented using someof the foregoing display technologies in a transparent or opticaltransmittive type, such as a transparent OLED (TOLED).

A rear configuration of the display module 151 can be implemented in theoptical transmittive type as well. In this configuration, a user can seean object at a rear portion of the mobile terminal 100 via an areaoccupied by the display module 151.

At least two display modules 151 can be provided in the mobile terminal100. For example, a plurality of display modules 151 can be arranged ona single face of the mobile terminal 100 spaced apart from each other orbuilt into one body. Alternatively, a plurality of display modules 151can be arranged on different faces of the mobile terminal 100.

If the display module 151 and the sensing unit 140 for detecting a touchaction (hereinafter called “touch sensor”) are configured as a mutuallayer structure (hereinafter called “touchscreen”), the display module151 can be used as a user input unit 130 as well as an output unit 150.The touch sensor can be configured as a touch film, a touch sheet, or atouchpad.

The touch sensor can be configured to convert a pressure applied to aspecific portion of the display module 151 or a variation of acapacitance generated from a specific portion of the display module 151to an electric input signal. Accordingly, the touch sensor detects apressure of a touch as well as a touched position or size.

If a touch input is made to the touch sensor, signal(s) corresponding tothe touch input is transferred to a touch controller (not shown). Thetouch controller processes the signal(s) and then transfers theprocessed signal(s) to the controller unit 180. Therefore, thecontroller unit 180 can determine whether a prescribed portion of thedisplay module 151 has been touched.

The proximity sensor 141 can be provided to an internal area of themobile terminal 100 enclosed by the display module 151, such as thetouchscreen or around the touchscreen. The proximity sensor 141 is asensor that detects a presence of an object approaching a prescribeddetecting surface or an object existing around the proximity sensor 141using an electromagnetic field strength or infrared ray withoutmechanical contact. The proximity sensor 141 can be more durable andmore useful than a contact type sensor.

The proximity sensor 141 can include a transmittive photoelectricsensor, a direct reflective photoelectric sensor, a mirror reflectivephotoelectric sensor, a radio frequency oscillation proximity sensor, anelectrostatic capacity proximity sensor, a magnetic proximity sensor, oran infrared proximity sensor. If the touchscreen includes theelectrostatic capacity proximity sensor, the touchscreen is configuredto detect the proximity of a pointer according to a variation in anelectric field formed by the proximity of the pointer to thetouchscreen. Accordingly, the touchscreen or touch sensor can beclassified as the proximity sensor 141.

An action when a pointer approaches without contacting the touchscreenso the pointer is recognized as being located on the touchscreen isdefined as “proximity touch.” An action when a pointer actually touchesthe touchscreen is defined as “contact touch.” The meaning of theposition on the touchscreen proximity-touched by the pointer means theposition of the pointer which vertically opposes the touchscreen whenthe pointer performs the proximity touch.

The proximity sensor 141 detects a proximity touch and a proximity touchpattern (e.g., a proximity touch distance, a proximity touch duration, aproximity touch position, or a proximity touch shift state.).Information corresponding to the detected proximity touch action and thedetected proximity touch pattern can be displayed on the touchscreen.

FIG. 2 is a conceptual diagram for explaining a proximity depth of aproximity sensor according to an embodiment of the present invention.When a pointer, such as a user's finger or a pen approaches the displaymodule 151, the proximity sensor 141 provided within or in the vicinityof the display module 151 detects the approach of the pointer and thenoutputs a proximity signal. The proximity sensor 141 can be configuredto output a different proximity signal according to a distance betweenthe pointer and the proximity-touched display module 151 (hereinafter“proximity depth”).

Referring to FIG. 2, a cross-section of the mobile terminal 100 isprovided with a proximity sensor 141 capable of sensing three proximitydepths, for example. It can be understood that the proximity sensor 141can be capable of sensing proximity depths amounting to a number smallerthan 3 and equal to or greater than 4.

For example, when the pointer is fully contacted with the display module151 at d0, the relation is recognized as a contact touch. When thepointer is spaced apart from the display module 151 at a distance d1,the relation is recognized as a proximity touch at a first proximitydepth. When the pointer is spaced apart from the display module 151 at adistance between d1 and d2, the relation is recognized as a proximitytouch at a second proximity depth. When the pointer is spaced apart fromthe display module 151 at a distance between d2 and d3, the relation isrecognized as a proximity touch at a third proximity depth. Further,when the pointer is spaced apart from the display module 151 at adistance equal to or greater than d3, no proximity touch is recognized.

Hence, the controller unit 180 can recognize the proximity touch as oneof various input signals according to the proximity depth and positionof the pointer relative to the display module 151. In some examples, thecontroller unit 180 can perform various operation controls according tothe various input signals.

Referring to FIG. 1, the audio output module 152 functions in variousmodes including a call-receiving mode, a call-placing mode, a recordingmode, a voice recognition mode, and a broadcast reception mode in orderto output audio data which is received from the wireless communicationunit 110 or stored in the memory unit 160. During operation, the audiooutput module 152 outputs audio relating to a particular function (e.g.,call received, message received). The audio output module 152 can beimplemented individually or by using one or more speakers, buzzers, andother audio producing devices.

The alarm module 153 outputs a signal for announcing an occurrence of aparticular event associated with the mobile terminal 100. Examples ofthe announced events can include a call received event, a messagereceived event and a touch input received event.

The alarm module 153 can output a signal for announcing the eventoccurrence by way of vibration as well as via a video or audio signal.The video or audio signal can be output via the display module 151 orthe audio output module 152. Hence, the display module 151 or the audiooutput module 152 can be regarded as a part of the alarm module 153.

The haptic module 154 generates various tactile effects that can besensed by a user. Vibration is a representative tactile effect generatedby the haptic module 154. Strength and pattern of the vibrationgenerated by the haptic module 154 can be controlled. For example,different vibrations can be output simultaneously or sequentially.

The haptic module 154 can generate various tactile effects as well asthe vibration. For example, the haptic module 154 generates an effectattributed to the arrangement of pins vertically moving against acontact skin surface, an effect attributed to the injection/suctionpower of air though an injection/suction hole, an effect attributed toskimming over a skin surface, an effect attributed to the contact withan electrode, an effect attributed to electrostatic force, or an effectattributed to the representation of hold/cold sense using an endothermicor exothermic device.

The haptic module 154 can be implemented to enable a user to sense thetactile effect through a muscle sense of a finger or an arm as well asto transfer the tactile effect through direct contact. At least twohaptic modules 154 can be provided in the mobile terminal 100.

Furthermore, the audio output module 152 can output sound through anearphone jack 156. The user can connect earphones to the earphone jack156 and hear the output sound.

The memory unit 160 can be used to store various types of data tosupport processing, control, and storage requirements of the mobileterminal 100. Examples of such stored data include program instructionsfor applications operating on the mobile terminal 100, contact data,phonebook data, messages, audio, still pictures, and moving pictures.The memory unit 160 can also store a recent use history or a cumulativeuse frequency of each data (e.g., use frequency for each phonebook, eachmessage or each multimedia). Moreover, data for various patterns ofvibration and/or sound output can be stored in the memory unit 160 whena touch input to the touchscreen is sensed.

The memory unit 160 may be implemented using any type or combination ofsuitable volatile and non-volatile memory or storage devices includinghard disk, random access memory (RAM), static random access memory(SRAM), electrically erasable programmable read-only memory (EEPROM),erasable programmable read-only memory (EPROM), programmable read-onlymemory (PROM), read-only memory (ROM), magnetic memory, flash memory,magnetic or optical disk, multimedia card micro type memory, card-typememory (e.g., SD memory, XD memory), or other similar memory or datastorage device. Further, the mobile terminal 100 can operate via a webstorage entity for performing a storage function of the memory unit 160on the Internet.

The interface unit 170 can be implemented to couple the mobile terminal100 with external devices. The interface unit 170 receives data from theexternal devices or is supplied with power and then transfers the dataor power to the respective elements of the mobile terminal 100 orenables data within the mobile terminal 100 to be transferred to theexternal devices. The interface unit 170 may be configured using awired/wireless headset port, an external charger port, a wired/wirelessdata port, a memory card port, a port for coupling to a device having anidentity module, audio input/output ports, video input/output ports, andan earphone port.

The identity module (not shown) can be an integrated circuit for storingvarious types of information for authenticating a use authority of themobile terminal 100 and can include a User Identify Module (UIM),Subscriber Identify Module (SIM), and Universal Subscriber IdentityModule (USIM). A device having the identity module (hereinafter called“identity device”) can be manufactured as a smart card. Therefore, theidentity device can be connected to the mobile terminal 100 via acorresponding port.

When the mobile terminal 100 is connected to an external cradle, theinterface unit 170 provides a passage for supplying power to the mobileterminal 100 from the external cradle or a passage for deliveringvarious command signals input by a user via the external cradle, to themobile terminal 100. Each of the delivered command signals input via theexternal cradle or the supplied power can signal that the mobileterminal 100 has been correctly loaded in the external cradle.

The controller unit 180 controls the overall operations of the mobileterminal 100. For example, the controller unit 180 controls andprocesses voice calls, data communications, and video calls.

The controller unit 180 may include a multimedia module 181 thatprovides multimedia playback. The multimedia module 181 may beconfigured as part of the controller unit 180, or implemented as aseparate component. Moreover, the controller unit 180 can perform apattern recognition process for recognizing characters of a writteninput and images of a picture drawing input carried out on thetouchscreen.

The power supply unit 190 provides power required by the variouscomponents of the mobile terminal 100. The provided power may beprovided internally or externally to the mobile terminal 100.

Various embodiments described herein may be implemented in acomputer-readable medium using, for example, computer software,hardware, or some combination thereof. For a hardware implementation,the embodiments described herein may be implemented individually orcombined within one or more application specific integrated circuits(ASICs), digital signal processors (DSPs), digital signal processingdevices (DSPDs), programmable logic devices (PLDs), field programmablegate arrays (FPGAs), processors, controllers, micro-controllers,microprocessors, or other electronic units designed to perform thefunctions described herein. The foregoing embodiments may also beimplemented by the controller unit 180.

For a software implementation, the embodiments described herein may beimplemented with separate software modules, such as procedures andfunctions, each of which perform one or more of the functions andoperations described herein. The software codes can be implemented witha software application written in any suitable programming language andmay be stored in memory such as the memory unit 160, and executed by acontroller or processor, such as the controller unit 180.

FIG. 3A is a perspective diagram of a front side of the mobile terminal100 according to an embodiment of the present invention. The mobileterminal 100 is configured to have a bar-type terminal body. Yet, themobile terminal 100 may be implemented in a variety of differentconfigurations. Examples of such configurations include a folder-type, aslide-type, a rotational-type, a swing-type and combinations thereof.For clarity, the remainder of the disclosure will discuss the mobileterminal 100 directed to the bar-type terminal body. However suchteachings apply equally to other types of mobile terminals.

Referring to FIG. 3A, the mobile terminal 100 includes a case (casing,housing, cover, etc.) configuring an exterior thereof. In the presentembodiment, the case can be divided into a front case 101 and a rearcase 102. Various electric or electronic parts are loaded in a spaceprovided between the front case 101 and rear case 102. Optionally, atleast one middle case can be additionally provided between the frontcase 101 and rear case 102.

The cases 101 and 102 can be formed by injection molding of syntheticresin or can be formed of metal substances such as stainless steel ortitanium.

The front case 101 of the mobile terminal 100 can include the displaymodule 151, the audio output unit 152, a camera module 121, user inputunits 131 and 132, the microphone module 122, or the interface unit 170.

The display module 151 occupies most of a main face of the front case101. The audio output unit 151 and the camera module 121 are provided onan area adjacent to one of two end portions of the display module 151,while the user input unit 131 and the microphone module 122 are providedto another area adjacent to the other end portion of the display module151. The user input unit 132 and the interface unit 170 can be providedon lateral sides of the front case 101 and rear case 102.

The input unit 130 is manipulated to receive a command for controllingan operation of the mobile terminal 100. The input unit 130 is able toinclude a plurality of manipulating units 131 and 132. The manipulatingunits 131 and 132 can be named a manipulating portion and may adopt anymechanism of a tactile manner that enables a user to perform amanipulation action by sensing a tactile feedback.

Content input by the first manipulating unit 131 or second manipulatingunit 132 can be diversely set. For instance, commands such as start, endor scroll can be input to the first manipulating unit 131. On the otherhand, commands directed to adjusting a volume of sound outputting fromthe audio output unit 152 or switching to a touch recognizing mode ofthe display module 151 can be input to the second manipulating unit 132.

FIG. 3B is a perspective diagram of a rear side of the mobile terminal100 shown in FIG. 4 according to an embodiment of the present invention.Referring to FIG. 3B, a camera module 121′ can be additionally providedto a backside of the mobile terminal 100, and more particularly, to therear case 102. The camera module 121′ has a photographing direction thatis substantially opposite to that of the camera module 121 shown in FIG.3A and may have a different number of pixels to provide a differentresolution from that of the camera module 121.

Preferably the camera module 121 can have enough resolution to captureand transmit a picture of a user's face for a video call, while thecamera module 121′ has a higher resolution for capturing a generalsubject for photography without transmitting the captured subject. Eachof the camera modules 121 and 121′ can be installed in the mobileterminal 100 to be rotated open or popped open.

The flash module 123 and the mirror module 124 are additionally providedadjacent to the camera module 121′. The flash module 123 projects lighttoward a subject in case of photographing the subject using the cameramodule 121′. In case that a user attempts to take a picture of the user(e.g., self-photography) using the camera module 121′, the mirror module124 enables the user to view user's face reflected by the mirror module124.

An additional audio output unit 152′ can be provided to the backside ofthe mobile terminal 100. The additional audio output unit 152′ is ableto implement a stereo function together with the audio output unit 152shown in FIG. 3A and may be used to implement a speakerphone mode whenthe mobile terminal 100 is configured in the phone call mode.

The broadcast signal antenna 116 can be provided to the lateral side ofthe mobile terminal 100 to provide further communication capabilities tothe mobile terminal 100. The broadcast signal antenna 116 can beconstructed as a portion of the broadcast receiving module 111 shown inFIG. 1. Additionally, the broadcast signal antenna 116 can be configuredto be retractable in the mobile terminal 100.

The power supply unit 190 for supplying power to the mobile terminal 100can be configured to be built within the mobile terminal 100.Alternatively, the power supply unit 190 can be configured to bedetachably connected to the mobile terminal 100.

A touchpad 135 for detecting a touch can be additionally provided to therear case 102. The touchpad 135 can be configured in a lighttransmittive type like the display module 151. In this case, if thedisplay module 151 is configured to output visual information from itsboth faces, it is able to recognize the visual information via thetouchpad 135 as well. The information output from both of the faces canbe entirely controlled by the touchpad 135. Alternatively, a display isfurther provided to the touchpad 135 so that a touchscreen can beprovided to the rear case 102 as well.

The touchpad 135 is activated by interconnecting with the display module151 of the front case 101. The touchpad 135 can be provided in rear ofthe display module 151 in parallel. The touchpad 135 can have a sizeequal to or smaller than that of the display module 151.

FIGS. 3C and 3D illustrate versions of the mobile terminal 100 accordingto an embodiment of the present invention. Referring to FIG. 3C, thedisplay module 151 can include a first display and a second displaywhich are physically separated from each other. In a folder type orslide type mobile terminal having two bodies connected through a hingeor slide, the first display (or main display) can be formed on the innerface or outer face of one of the bodies and the second display (or subdisplay) can be formed on the inner face or outer face of the otherbody. The sub display is separated from the mobile terminal anddetachably combined with the mobile terminal body through an interfaceto display data from the mobile terminal 100. Referring to FIG. 3D, thedisplay module 151 can include a first display 350 and second display360 which are logically partitioned from each other.

Referring to FIG. 4, a CDMA wireless communication system 400 includesmobile terminals 100, base stations 470, base station controllers 475,and a mobile switching center 480. The mobile switching center 480 isconnected to a public switch telephone network (PSTN) 490. The mobileswitching center 480 is connected to the base station controllers 475.The base station controllers 475 are connected to the base stations 470through backhaul lines. The backhaul lines may be constructed accordingto at least E1/T1, ATM, IP, PPP, frame relay, HDSL, ADSL or xDSL. TheCDMA wireless communication system 400 may include at least two basestation controllers 475.

Each base station 470 may include one or more sectors and each sectormay include an omnidirectional antenna or an antenna adjusted to aspecific radiation direction from the base station 470. Alternatively,each sector may include two diversity reception antennas. Each basestation 470 is constructed to have frequency assignments, and thefrequency assignments may have specific spectra (e.g., 1.25 MHz and 5MHz). Furthermore, an intersection of sectors and frequency assignmentsmay be referred to as a CDMA channel.

The base stations 470 may be referred to as base station transceiversubsystems (BTSs). A “Base station” may be used as a term thatcollectively designates the base station controller 475 and one or morebase stations 470 in several examples. Furthermore, the base stations470 may be referred to as “cell sites.” Alternatively, individualsectors of a given base station 470 may be referred to as cell sites.

A terrestrial DMB transmitter 495 can transmit broadcasting signals tothe mobile terminals 100 operating in the CDMA wireless communicationsystem 400. The broadcasting receiving module 111 (FIG. 1) of eachmobile terminal 100 is constructed to receive the broadcasting signalstransmitted from the DMB transmitter 495. This can be similarly appliedto different types of broadcast and multicast signaling as describedabove.

The CDMA wireless communication system 400 may communicate with globalpositioning system (GPS) satellites 410. The GPS satellites 410 cantrack the positions of some or all of the mobile terminals 100 in thesystem. Although two satellites are shown in FIG. 4, positioninformation can be obtained from less than or more than two satellites.In addition, other position-tracking techniques, such asposition-tracking techniques that can substitute for GPS technique orcan be added to the GPS technique, can be used. If required, some or allof the GPS satellites 410 can support satellite DMB transmissionseparately or additionally.

When the CDMA wireless communication system 400 operates, the basestations 470 receive reverse link signals from the mobile terminals 100.The mobile terminals 100 may be in a state that the mobile terminals 100are making calls, sending messages or performing other communications.The reverse link signals received by the base stations 470 are processedby the base stations 470. The processed data is transmitted to the basestation controllers 475 connected to the base stations 470. The basestation controllers 475 provide call resource allocation and mobilitymanagement functionality including soft handoffs between the basestations 470. Furthermore, the base station controllers 475 transmit thereceived data to the mobile switching center 480. The mobile switchingcenter 480 provides additional routing services for interfacing with thePSTN 490. Similarly, the PSTN 490 interfaces with the mobile switchingcenter 480, and the mobile switching center 480 interfaces with the basestation controllers 475. The base station controllers 475 control thebase stations 470 to transmit forward link signals to the mobileterminals 100.

FIG. 5 is a flowchart showing an operation of the mobile terminalaccording to an embodiment of the present invention. The controller unit180 (FIG. 1) of the mobile terminal 100 may control the camera module121 to photograph an image (S10). The controller unit 180 may acquiregeographic information corresponding to the photographed image (S20).The controller unit 180 may acquire information about an object includedin the photographed image (S30). The controller unit 180 may set aspecific time in relation to the photographed image (S40). Finally, whenthe specific time is set, the controller unit 180 may display theinformation about the object at the specific time (S50).

FIGS. 6A and 6B are views describing the operation of the mobileterminal according to an embodiment of the present invention. Referringto FIG. 6A, the camera module 121 may be activated by an operation ofthe user via the mobile terminal 100 or a control operation of thecontroller unit 180. The activated camera module 121 can photograph asurrounding video “SV.” The surrounding video “SV” may depend on theorientation of the mobile terminal 100. For example, the camera module121 captures an image or moving image depending on the orientation ofthe mobile terminal 100 even though the camera module 121 is activatedat a specific point.

FIG. 6B illustrates an object “OB” of a theater and information aboutthe object “OB” as the present program of the theater. Additionally,information may be displayed through a pop-up window “P.” Referring toFIG. 6B, the controller unit 180 can acquire geographic informationcorresponding to the photographed image (S20 of FIG. 5). The geographicinformation may be elementary information used to obtain informationabout the surrounding picture “SV” captured by the camera module 121.Geographic information about the position and photographing direction ofthe mobile terminal 100 may be required to acquire information about anobject “OB” included in the surrounding video “SV” photographed by thecamera module 121. For example, if there is fundamental geographicinformation representing that the mobile terminal 100 is photographingan image in a direction at an angle of 15° to the north from a point (x,y), the controller unit 180 can recognize the object “OB” located in thedirection to acquire the information about the object “OB.”

In one embodiment, the geographic information may be obtained through aglobal navigation satellite system (GNSS), for example. The GNSSacquires geographic information using a method of obtaining signalstransmitted from GPS satellites 410 in orbit (FIG. 4). For instance, theposition information module 115 of the mobile terminal 100 detects theposition of the mobile terminal 100 using a time difference amongsignals transmitted from the GPS satellites 410. The GNSS is classifiedinto global positioning system (GPS), Galileo, global orbitingnavigation satellite system (GLONASS), Indian regional navigationsatellite system (IRNSS) and Quasi-Zenith satellite system (QZSS)according to the central operating body.

In another embodiment, the geographic information may be acquiredthrough a geomagnetic sensor included in the mobile terminal 100. Thegeomagnetic sensor senses terrestrial magnetism penetrating the northand south poles of the earth and functions as a kind of a compass.

In still another embodiment, the geographic information may be acquiredthrough an acceleration sensor module 141 (not shown) included in thesensing unit 140 of the mobile terminal 100. The acceleration sensor cansense a relative position of the mobile terminal 100 based on a specificpoint (x, y) in a three-dimensional space.

The controller unit 180 may recognize the position of the mobileterminal 100 on the earth and the orientation of the camera 100 in theposition from the geographic information acquired using the GNSS,geomagnetic sensor or acceleration sensor.

The object “OB” may be a target from which information is acquired. Thatis, the object “OB” may be a specific subject that is included in thesurrounding video “SV” captured by the camera module 121 and hasinformation in which the user is interested in retrieving. The object“OB” may include material and immaterial objects. The material objectmay include at least, but not limited to, buildings or roads and theimmaterial object may include weather, for example.

Since the geographic information including the orientations of themobile terminal 100 and the camera module 121 has been acquired, thecontroller unit 180 can determine the object “OB” captured by the cameramodule 121. For example, the controller unit 180 knows that the cameramodule 121 is taking a picture in a direction A from the point (x, y),and thus the controller unit 180 can recognize the object “OB” existingin the direction A.

The controller unit 180 may specify the object “OB” through geographicinformation and image analysis. In some examples, the surrounding video“SV” captured by the camera module 121 may include various objects inaddition to the specific object “OB” in which the user is interested.When the user selects the specific object “OB” from the various objectsincluded in the surrounding video “SV,” the controller unit 180 canhandle the image of the object “OB” separately from the other objects.The image of the object “OB” may be analyzed by the controller unit 180or transmitted to an external server (not shown) and analyzed by theexternal server.

When the specific object “OB” included in the surrounding video “SV” isspecified, information about the object “OB” may be acquired Thecontroller unit 180 may acquire information about the specific objectincluded in the photographed image (S30 of FIG. 5). The external servermay be requested to provide the information about the object “OB.”Alternatively, information previously stored in the memory unit 160 ofthe mobile terminal 100 may be used as the information about the object“OB.”

The information about the object “OB” may vary with time. If the object“OB” is a building, for example, another building might be located atthe site of the building corresponding to the object “OB” at a firstspecific time in the past and the building corresponding to the object“OB” might be under construction at a second specific time in the past.If the object “OB” is a theater, different programs might be performedat the theater at a specific time in the past, in the present or at aspecific time in the future. The mobile terminal 100 according to anembodiment of the present invention can extract only informationcorresponding to a specific time from the information about the object“OB,” which may vary with time, and set the specific time according to auser's choice or based on a control signal of the controller unit 180.

The controller unit 180 may set the specific time through a timeline“TL” displayed on the display module 151. The timeline “TL” mayrepresent the past by a left portion thereof and represent the future bya right portion thereof relative to a cursor designated as present timeon the timeline “TL.” The mobile terminal 100 may display informationabout the object “OB” at the present time as a default setting. If theuser drags or flicks the timeline “TL” to the left for a specific timein the past or to the right for a specific time in the future may beselected based on the direction and intensity of the dragging orflicking operation.

The information about the object “OB” may depend on the selected object“OB.” If the selected object “OB” is a building, for example, theinformation about the object “OB” may include the constructing processof the building or price information. The information about the object“OB” may include the contents of the performance presented at a theaterif the object “OB” is a theater, weather if the selected object is skyand stock information if the object “OB” is the logo of a specificpublicly traded company. The user or the controller unit 180 maydetermine the information to be displayed.

FIG. 7 is a flowchart illustrating an operation of capturing an imageaccording to an embodiment of the present invention. Referring to FIG.7, the operation S10 of capturing an image may include activating thecamera module 121 (S11). The camera module 121 may not be activatedgenerally. That is, the camera module 121 may not be configured in astate to photograph images by default. When the camera module 121 isactivated and enters a standby state, it may be necessary to supplypower to the camera module 121 including charge coupled device (CCD)elements. Accordingly, the controller unit 180 can reduce power suppliedfrom the power supply unit 190 to the camera module 121 such that thecamera module 121 transitions to an inactive state. The camera module121 in the inactive state can be activated by an operation of the useror a control operation of the controller unit 180. That is, the cameramodule 121 transitions to a state for photographing images (e.g.,pictures) or moving images (e.g., video). When the camera module 121 isactivated to begin photographing an image, the display module 151 maydisplay the image captured by the camera module 121. Furthermore, whenthe camera module 121 captures the image, an object image “IM” may beextracted (S12).

FIGS. 8A and 8B are views for explaining the operation of the mobileterminal according to an embodiment of the present invention. Referringto FIG. 8A, when the camera module 121 captures the image, an objectimage “IM” may be extracted from the captured image (S12 of FIG. 7).

The object image “IM” is an image of the object “OB” having informationin which the user is interested among the various objects included inthe captured image. The image captured by the camera module 121 anddisplayed on the display module 151 may include various objects inaddition to the object “OB” in which the user is interested. When thespecific object “OB” is selected from the image displayed on the displaymodule 151 according to an operation of the user or a control operationof the controller unit 180, the selected object “OB” may be handleddifferently from the other objects. For example, the user can touch theoutline of the object “OB” in the form of a looped curve or touch withinthe boundary of the object “OB” to select the specific object “OB.”

Referring to FIG. 8B, upon the selection of the object “OB,” thecontroller unit 180 may go to the operation S20 (FIG. 5) of acquiringthe geographic information about the object “OB,” or transmit the objectimage IM to the server. The server, for example, can analyze the objectimage “IM” using resources more abundant than the mobile terminal 100.

FIG. 9 is a flowchart illustrating an operation of acquiring geographicinformation according to an embodiment of the present invention.Referring to FIG. 9, the operation S20 (FIG. 5) of acquiring thegeographic information may include acquiring information about aphotographing point “TP” (S21). Additionally, the operation S20 mayfurther include acquiring information about a photographing direction“TD” (S22).

FIG. 10 is a visual representation of acquired geographic informationfor operating the mobile terminal 100 according to an embodiment of thepresent invention. The geographic information about a photographingpoint “TP” and a photographing direction “TG” may be included in basicdata used to determine the object “OB” in which the user is interested.That is, the information about the photographing point “TP” and thephotographing direction “TG” can be used with the object image “IM”(FIG. 8).

The photographing point “TP” and the photographing direction “TG” may beimportant basic data used to recognize the area photographed by thecamera module 121. That is, the object of the camera module 121 can bechanged if any one of the photographing point “TP” and the photographingdirection “TG” varies. Accordingly, the controller unit 180 can analyzethe photographing point “TP” and the photographing direction “TG” withrespect to the object image “IM,” or transmit the photographing point“TP” and the photographing direction “TG” with the object image “IM” tothe server. The geographic information about the photographing point“TP” may be acquired through GNSS and the geographic information aboutthe photographing direction “TG” may be obtained through a geomagneticsensor.

FIG. 11 is a flowchart illustrating the operation S30 of acquiringinformation about an object included in a captured image according to anembodiment of the present invention. Referring to FIG. 11, the operationS30 of acquiring the information about the object “OB” included in thesurrounding video SV (FIG. 5) may include determining whether theinformation about the object “OB” exists in memory (S31). In oneembodiment, the information about the object “OB” may be stored in thememory unit 160 of the mobile terminal 100. For example, the user canretrieve information about the object “OB” from the memory unit 160 orfrom a database related to the mobile terminal 100.

If the object information exists in memory, then the controller unit 180may load the information about the object “OB” from the memory unit 160(S35).

If the information about the object “OB” does not exist in the memoryunit 160, the controller unit 180 may transmit the geographicinformation about the photographing point (“TP” shown in FIG. 10), thegeographic information about the photographing direction (“TD” shown inFIG. 10) and the object image (“IM” shown in FIG. 8) to a server 1000(S32), receive information about the object “OB” from the server 1000(S33), and store the received information in the memory unit 160 (S34).

The photographing point “TP,” the photographing direction “TD” and theobject image “IM” may be basic data used to recognize the object “OB.”The controller unit 180 may transmit the whole or part of theinformation about the photographing point “TP,” the information aboutthe photographing direction “TD” and the object image “IM” to the server1000. For example, the object “OB” can be recognized only using theobject image “IM” through an image analysis method.

Furthermore, the object “OB” can be recognized more correctly by usingthe information about the photographing point “TP” and the photographingdirection “TD.” The controller unit 180 can transmit the whole or partof the information about the photographing point “TP,” the informationabout the photographing direction “TD” and the object image “IM” to theserver 1000 such that the server 1000 can analyze the photographingpoint “TP,” the photographing direction “TD” and the object image “IM.”

FIG. 12 is a flowchart illustrating the operation of the mobile terminalaccording to an embodiment of the present invention. Referring to FIG.12, the server 1000 receives the information from the mobile terminal100 and searches for the information about the object “OB” based on thereceived information (S36). The server 1000 may contain more resourcesthan the mobile terminal 100 to be suitable for acquiring informationand processing the acquired information.

The controller unit 180 of the mobile terminal 100 may receive thesearched information about the object “OB” from the server 1000 (S33 ofFIG. 11) and store the received information in the memory unit 160 (S34of FIG. 11). If the information is stored in the memory unit 160, thereis no need to repeatedly receive the information about the object “OB.”

FIG. 13 is a flowchart illustrating an operation of setting a specifictime according to an embodiment of the present invention. Referring toFIG. 13, the operation S40 of setting the specific time may includedisplaying a timeline “TL” (S41). The timeline “TL” may function as aninterface through which the user can input a specific time. That is, theuser can touch the timeline “TL” in a first direction or in a seconddirection to select information corresponding to a time on the timeline“TL” to be displayed on the display module 151. For example, the usercan flick the timeline “TL” to the left. In this case, information to bedisplayed on the display module 151 may be information corresponding toa specific time in the future.

The timeline “TL” may indicate a time at which the state of the object“OB” changes (S42). For example, if a present time is 1 p.m., it raineduntil 9 a.m., the weather is clear from 9 a.m. up to the present time,and it will rain at 9 p.m., the weather corresponding to the object “OB”should indicate changes at 9 a.m. and 9 p.m. In the case of weatherinformation, the weather at a specific time and a time at which theweather changes may be important information that draws the user'sattention. Accordingly, if the timeline “TL” indicates the time at whichthe state of the object “OB” changes, the user can conveniently find outthe time.

Subsequently, the controller unit 180 may receive a touch input appliedto the timeline “TL” (S43). The touch input applied to the timeline “TL”may be dragging or flicking the timeline “TL” to the left or to theright. Upon the acquisition of the touch input applied to the timeline“TL,” the controller unit 180 may move an indicator of the timeline “TL”when the state of the object “OB” changes (S44).

The timeline “TL” may be displayed proportionately with the displaymodule 151. For example, the timeline “TL” may be displayed in arelatively small size due to a small area of the display module 151 ofthe mobile terminal 100. In this case, the user may have a difficulty incorrectly operating the timeline “TL” when touching the timeline TL.Accordingly, the controller unit 180 can move the indicator of thetimeline “TL” when the state of the object “OB” changes to improve theconvenience of use. For instance, if the weather changes at 9 a.m. and 9p.m., as described above, the indicator is moved from the present timeof 1 p.m. to 9 a.m. when the user flicks the timeline “TL” to the left(i.e., to a future time). Alternatively, the indicator can be moved fromthe present time 1 p.m. to 9 p.m. when the user flicks the timeline “TL”to the right (i.e., to a past time). The operation is performed based onthe time at which the state of the object “OB” in which the user isinterested changes, and thus an erroneous operation can be prevented.

The controller unit 180 may determine whether or not the time settingoperation ended (S45) and perform the next operation when the timesetting operation ended.

FIGS. 14 through 19 are timelines displayed via the display module 151of the mobile terminal 100 according to an embodiment of the presentinvention. Referring to FIG. 14, the timeline “TL” may include a firstindicator “PR” indicating a present time. In addition, the timeline “TL”may include a second indicator “DP” indicating a reference of the imagebeing displayed on the display module 151. Since the first indicator“PR” is located at the second indicator “DP,” the image currentlycaptured by the camera module 121 is being displayed on the displaymodule 151.

Moreover, first “I1,” second “I2” and third icon “I3” may indicatemoments of time at which the state of the object “OB” changes. Since thefirst icon “I1” is located to the left of the first indicator “PR” andthe second icon “I2” and third icon “I3” are placed to the right of thefirst indicator “PR,” the user can be notified that the state of theobject “OB” changed at a specific time in the past (i.e., indicated bythe first icon “I1”) and the state of the object “OB” will change atspecific moments of time in the future (i.e., indicated by the secondicon “I2” and third icon “I3”).

Furthermore, first arrow “PA” indicating a past time and second arrow“FU” indicating a future time allow the user to intuitively know whetherthere is an event of changing the state of the object “OB” in the pastor in the future. The first arrow “PA” and second arrow “FU” aredisplayed according to a change in the state of the object “OB” in thepast time and the future time.

Finally, the user may touch the timeline “TL” in a first direction “D1”or a second direction “D2” with a finger “F.” Touching the timeline “TL”in the first direction “D1” means moving to a specific time in the pastand touching the timeline “TL” in the second direction “D2” means movingto a specific time in the future. That is, the user can drag or flickthe timeline “TL” in the first direction “D1” when the user wants tomove to the past time from the present time. Alternative, the user maydrag or flick the timeline “TL” in the second direction D2 when the userwants to move to the future time from the present time.

Referring to FIG. 15, when the user touches the timeline “TL” in thefirst direction (“D1” shown in FIG. 14), the display of the timeline“TL” is moved to the left. That is, the first indicator “PR” indicatingthe present time is moved to the left of the second indicator “DP”indicating the reference of the image displayed on the display module151. The display of the timeline “TL” can vary according to when thestate of the object “OB” changes. The second icon “I2” can be located bythe second indicator “DP,” and thus the state of the object “OB” at aspecific time (i.e., indicated by the second icon “I2”) can be displayedon the display module 151.

Referring to FIG. 16, when the user touches the timeline “TL” in thesecond direction (“D2” shown in FIG. 14), the first indicator “PR” ismoved to the right of the second indicator “DP.” Furthermore, the firsticon “I1” representing a specific time at which the state of the object“OB” changed in the past can be located by the second indicator “DP”since the first indicator “PR” moves according to when a specific eventof changing the state of the object “OB” is generated. In addition, thetimeline “TL” further includes a fourth icon “I4” indicating that anevent of changing the state of the object “OB” was generated at a timeprior to the specific time corresponding to the first icon “I1” in thepast.

FIG. 17 shows a relationship between the length of a user's touch and amoving distance of the timeline “TL.” Referring to FIG. 17, the user maytouch the timeline “TL” from a point “TP” in a leftward direction withdifferent touch lengths. That is, the user may touch the timeline “TL”from the point “TP” to the left in a first touch length “D11” that doesnot reach the second indicator “DP” indicating the reference of theimage displayed on the display module 151, a second touch length “D12”reaching the second indicator “DP” and a third touch length “D13” thatpasses the second indicator “DP.”

Here, the second icon “I2” is moved to the second indicator “DP” for allthe three cases although the user's touches have different lengths. Thatis, the timeline “TL” according to an embodiment of the presentinvention can move the indicators when the state of the object “OB”changes, and thus information corresponding to a next event can bedisplayed on the display module 151 if there is a user's touch toward aspecific direction. The event means a change in the state of the object“OB.” When there is a user's touch toward a specific direction, theinformation corresponding to the next event can be displayed on thedisplay module 151 irrespective of the length of the user's touch sothat the user can conveniently operate the mobile terminal 100.

FIGS. 18 and 19 illustrate a first arrow “PA” and a second arrow “FU”being displayed according to whether an event is generated indicating achange of state of the object “OB.” Referring to FIG. 18, a fifth icon“I5” is located on a point corresponding to a future time of thetimeline “TL” based on the second indicator “DP.” That is, the fifthicon “I5” shows that there is an event of changing the state of theobject “OB” at a specific time in the future. To clearly inform the userof the event, the timeline “TL” may display the second arrow “FU.” Theuser can intuitively know there is information about a specific time inthe future from the second arrow “FU.” Meantime, information about anevent generated in a past time based on the second indicator “DP” maynot exist. In this case, the first arrow “PA” is not displayed.

Referring to FIG. 19, a sixth icon “I6” representing that an event isgenerated at a specific time in the past may be displayed on thetimeline “TL,” and thus the first arrow “PA” can be displayed.Furthermore, when there is no information about a future event, thesecond arrow “FU” is not displayed.

FIG. 20 is a flowchart illustrating an operation of displaying objectinformation at a set time according to an embodiment of the presentinvention. Referring to FIG. 20, the operation S50 (FIG. 5) ofdisplaying the object information of the set time may includedetermining whether the object captured by the camera module 121 haschanged (S51). If the object “OB” captured by the camera module 121 hasnot changed, the controller unit 180 may display the acquiredinformation on the display module 151 (S52). When the acquiredinformation is displayed on the display module 151, the controller unit180 may determine whether the user selects a specific link (S53).Thereafter, the user may move to the selected link (S54). Otherwise, thecontroller unit 180 determines again whether the object has changed whenthe specific link was not selected (S55).

FIGS. 21 through 23 are conceptual views illustrating operation ofdisplaying object information at a set time via the display module 151of the mobile terminal 100 according to an embodiment of the presentinvention. A user can vary the position and orientation of the mobileterminal 100 to change the object “OB” captured by the camera module 121frequently. A method of displaying information relating to an imagecaptured by the user and displayed may be referred to as augmentedreality (AR). The controller unit 180 may display the informationrelating to the object “OB” using the “AR.” Accordingly, if the object“OB” of the camera module 121 is changed, the operations of capturing animage of the changed object and acquiring information about the changedobject can be repeated.

Referring to FIG. 21, if the photographing direction is changed from afirst orientation “TD1” to a second orientation “TD2” even though thephotographing point “TP” is not changed, the camera module 121 canphotograph a different object. The change of the photographing directionfrom the first orientation “TD1” to the second orientation “TD2” may besensed by a geomagnetic sensor, for example.

If the object “OB” of the camera module 121 is not changed, thecontroller unit 180 may display the acquired information on the displaymodule 151 (S52 of FIG. 20). The information about the object “OB” canbe acquired through the aforementioned operation S30 shown in FIG. 5.

Referring to FIG. 22, the acquired information may be displayed througha first pop-up window “P1.” When the object “OB” is a theater,information about the theater is displayed through the first pop-upwindow “P1.” For example, a title or a period relating to a programmingschedule of the theater can be displayed. Furthermore, buttons “B”connected to related functions may be displayed on the first pop-upwindow “P1.” For example, a first button “B1” connected to a link forobtaining detailed information and a second button “B2” connected to alink for buying a ticket can be displayed on the first pop-up window“P1.”

The information about the object “OB,” displayed via the display module151, may be changed in response to a user's touch input. That is, pastor future information about the object “OB” can be displayed in responseto a touch of the timeline “TL.” For example, the user touches thetimeline “TL” and the controller unit 180 displays information of theobject “OB” with respect to a time period corresponding to the touch.

Referring FIG. 23, when the second icon “I2” is located on the secondindicator “DP” in response to a user's touch, information with respectto a time period corresponding to the user's touch is displayed througha second pop-up window “P2.”

When the information is displayed on the display module 151, thecontroller unit 180 may determine whether the user selects a specificlink (S53 of FIG. 20) and move to the selected link (S54 of FIG. 20).The specific link may be represented by the first and second buttons“B1” and “B2.” When the user selects one of the first and secondbuttons, the user can move to a web page including informationcorresponding to the selected button.

Subsequently, the controller unit 180 may determine whether the objectof the camera module 121 changed again (S55 of FIG. 20). If the objectcaptured by the camera module 121 has not changed, the informationdisplayed on the display module 151 can be maintained. If the objectchanged, the operation of acquiring new information can be repeated.

FIGS. 24A through 27B are views for explaining an operation of themobile terminal according to an embodiment of the present invention. Thecontroller unit 180 can provide various information items in variousmanners.

Referring to FIG. 24A, the controller unit 180 may control the displaymodule 151 to display a present appearance “OB1” of the object.Specifically, the display module 151 may display the image currentlycaptured by the camera module 121. The image currently photographed canbe displayed via the display module 151, and thus the first indicator“PR” is superimposed with the second indicator “DP” on the timeline“TL.”

Moreover, a first icon “I1” can be displayed at a specific time in thepast on the timeline “TL.” That is, the object corresponding to thespecific time in the past has a different state from the current state“OB1.”

Referring to FIG. 24B, the user may touch the timeline “TL” to locatethe first icon “I1” indicating the specific time in the past on thesecond indicator “DP.” Then, the past object “OB2” corresponding to thefirst icon “I1” may be displayed on the display module 151. The eventcorresponding to the first icon “I1” represents the past object “OB2”under construction.

In addition, a second icon “I2” indicating a time prior to the timeindicated by the first icon “I1,” may be displayed on the display module151. That is, there may be information about a building existing beforethe past object “OB2.” The existence of the information about the pastobject can be known through manipulation of the first arrow “PA.”

Referring to FIGS. 25A and 25B, the controller unit 180 may displayweather information on the display module 151. Referring to FIG. 25A,the first, second and third icons (e.g., “I1,” “I2” and “I3”) may bedisplayed on the timeline “TL.” The icons “I1,” “I2” and “I3” mayrepresent weather information, particularly, moments of time at whichthe weather condition changes. In a first example, the weather before atime indicated by the first icon “I1” may be different from the weatherduring a period of time indicated by the first and second icons (e.g.,“I1”, “I2”). In a second example, the weather before the time indicatedby the second icon “I2” may be different from the weather during aperiod of time indicated by the second and third icons (e.g., “I2” and“I3”). In a third example, the weather during the period indicated bythe second and third icons may be different from the weather existingafter the time indicated by the third icon “I3.” Past weatherinformation represents actual weather, and future weather informationrepresents forecasted weather.

Referring to FIG. 25B, the user may touch the timeline “TL” to confirmthe weather of a specific time in the future. That is, the user cantouch the timeline “TL” to locate the third icon “I3” on the secondindicator “DP.” Then, weather information corresponding to the thirdicon “I3” is displayed on the display module 151. If the weatherinformation corresponding to the third icon “I3” is “rain,” an imageexpressing the “rain” may overlap the image captured by the cameramodule 121 and displayed on the display module 151. Furthermore, apop-up window (not shown) may be displayed on the display module 151 totransmit detailed weather information in text.

Referring to FIGS. 26A and 26B, the controller unit 180 may displaystock information of a specific publicly traded company on the displaymodule 151. Referring to FIG. 26A, an image of a street may be capturedby the camera module 121 and displayed on the display module 151. Thedisplayed image may include an advertisement “BC” corresponding to aspecific company. The user may touch the advertisement “BC.”

Referring to FIG. 26B, the controller unit 180 may analyze the image ofthe advertisement “BC” in response to the touch signal. Furthermore, thecontroller unit 180 may transmit the image of the advertisement “BC” toa server such that the server analyzes the image of the advertisement.When the controller unit 180 confirms the advertisement of the specificcompany from the analysis, the controller unit 180 may request for thestock information of the company and acquire the stock information. Thecontroller unit 180 may display the acquired stock information through apop-up window “P.” Furthermore, the controller unit 180 may operate thetimeline “TL” to display stock information corresponding to a specifictime in the present or past.

Referring to FIGS. 27A and 27B, the controller unit 180 can displaytraffic information on the display module 151. Referring to FIG. 27A, aroad may be photographed through the camera module 121 and an image ofthe road may be displayed on the display module 151. The controller unit180 may provide traffic information of the road with the image of theroad. That is, the controller unit 180 may provide information about theroad such as whether the road is ‘congested’, ‘normal’ or ‘smooth’ usingdifferent signs or colors such that the user can intuitively recognizetraffic on the road. The first indicator “PR” can be located on thesecond indicator “DP” on the timeline “TL.” In other words, the currenttraffic can be displayed using the first indicator “PR.”

Referring to FIG. 27B, the user may touch the timeline “TL” to check thetraffic of the same road at a specific time in the past. Here, thesecond indicator “DP” indicates the time as “12 hours” prior to thefirst indicator “PR.” Accordingly, the display module 151 can displaytraffic information directed to specific time of “12 hours” prior.

The above-described method of controlling the mobile terminal may bewritten as computer programs and may be implemented in digitalmicroprocessors that execute the programs using a computer-readablerecording medium. The method of controlling the mobile terminal may beexecuted through software. The software may include code segments thatperform required tasks. Programs or code segments may also be stored ina processor readable medium or may be transmitted according to acomputer data signal combined with a carrier through a transmissionmedium or communication network.

The computer readable recording medium may be any data storage devicethat can store data that can be thereafter read by a computer system.Examples of the computer readable recording medium may include read-onlymemory (ROM), random-access memory (RAM), CD-ROMs, DVD±ROM, DVD-RAM,magnetic tapes, floppy disks, and optical data storage devices. Thecomputer-readable recording medium may also be distributed over anetwork coupled computer systems so that the computer-readable code isstored and executed in a distributed fashion.

A mobile terminal may include a first touch screen configured to displaya first object, a second touch screen configured to display a secondobject, and a controller configured to receive a first touch inputapplied to the first object and to link the first object to a functioncorresponding to the second object when receiving a second touch inputapplied to the second object while the first touch input is maintained.

A method may be provided of controlling a mobile terminal that includesdisplaying a first object on the first touch screen, displaying a secondobject on the second touch screen, receiving a first touch input appliedto the first object, and linking the first object to a functioncorresponding to the second object when a second touch input applied tothe second object is received while the first touch input is maintained.

Any reference in this specification to “one embodiment,” “anembodiment,” “example embodiment,” etc., means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of this document. Theappearances of such phrases in various places in the specification arenot necessarily all referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with any embodiment, it is submitted that it is within thepurview of one skilled in the art to reasonably understand such feature,structure, or characteristic in connection with other embodiments.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, various variations and modificationsare possible in the component parts and/or arrangements of the subjectcombination arrangement within the scope of the disclosure, the drawingsand the appended claims. In addition to variations and modifications inthe component parts and/or arrangements, alternative uses will also beapparent to those skilled in the art.

1. A method comprising: displaying an image on a display of a mobileterminal, wherein the image depicts an object; displaying informationsimultaneously with the image, wherein the information relates to theobject at a particular time point of any of a plurality of differenttime points of a timeline; and changing the displayed information toreflect a change in the particular time point to another time point ofthe plurality of different time points of the timeline, wherein thechanged displayed information is displayed simultaneously with the imageand relates to the object at the another time point.
 2. The method ofclaim 1, wherein the image is obtained via a camera of the mobileterminal, a memory of the mobile terminal, or a wireless communicationunit of the mobile terminal.
 3. The method of claim 1, wherein theinformation is obtained from a server or a memory of the mobileterminal.
 4. The method of claim 1, further comprising: displaying auser interface with respect to the timeline; and receiving a user inputvia the user interface to select the particular time point or theanother time point.
 5. The method of claim 1, wherein displaying theuser interface comprises: displaying at least one indicator relative tothe displayed user interface to indicate a specific time point at whicha state of the depicted object changes.
 6. The method of claim 1,further comprising: changing the depicted object to a particular statecorresponding to the another time point; and displaying the changeddepicted object simultaneously with the changed displayed information.7. The method of claim 1, further comprising: obtaining geographicinformation relating to the displayed object.
 8. The method of claim 7,wherein the obtained geographic information relates to a generallocation at which the displayed object is photographed.
 9. The method ofclaim 7, wherein the obtained geographic information relates to at leasta photographing point or a photographing direction.
 10. The method ofclaim 7, wherein displaying the image comprises: activating a camera inthe mobile terminal; and extracting the depicted object from the image.11. The method of claim 10, further comprising: determining whether theinformation is stored in a memory of the mobile terminal; transmittingthe obtained geographic information and information identifying theextracted object to a server when the information is not stored in thememory; receiving the information relating to the extracted object fromthe server; and storing the received information in the memory.
 12. Themethod of claim 1, wherein displaying the information comprises:displaying a specific link; determining whether the displayed specificlink is selected; and displaying information corresponding to theselected specific link.
 13. A method of controlling information in amobile terminal, the method comprising: displaying an image on a displayof a mobile terminal, wherein the image depicts an object; displayinginformation simultaneously with the image, wherein the informationrelates to the object at a particular time point of any of a pluralityof different time points of a timeline; and changing the depicted objectto a different version to reflect a change in the particular time pointto another time point of the plurality of different time points of thetimeline, wherein the different version of the depicted object generallycorresponds to a state of the object at the another time point, andwherein the changed depicted object is displayed simultaneously with theinformation.
 14. A method of controlling information in a mobileterminal, the method comprising: displaying an image on a display of amobile terminal, wherein the image depicts an object; displayinginformation simultaneously with the image, wherein the informationrelates to the object at a particular time point of any of a pluralityof different time points of a timeline; changing the displayedinformation to reflect a change in the particular time point to anothertime point of the plurality of different time points of the timeline;and changing the depicted object to reflect the object at the anothertime point, wherein the changed displayed information is displayedsimultaneously with the changed depicted object.
 15. A mobile terminalcomprising: a display; and a controller configured to: display an imageon the display, wherein the image depicts an object; display informationsimultaneously with the image, wherein the information relates to theobject at a particular time point of any of a plurality of differenttime points of a timeline; and change the displayed information toreflect a change in the particular time point to another time point ofthe plurality of different time points of the timeline, wherein thechanged displayed information is displayed simultaneously with the imageand relates to the object at the another time point.
 16. The mobileterminal of claim 15, further comprising a wireless communication unitconfigured to receive the information.
 17. The mobile terminal of claim16, further comprising a memory configured to store the receivedinformation.
 18. The mobile terminal of claim 15, wherein the controlleris further configured to: display a user interface corresponding to thetimeline on the display; and receive a user input for selecting theparticular time point via the user interface.
 19. The mobile terminal ofclaim 18, wherein the user input is a dragging touch input or a flickingtouch input in either a first direction or a second direction in whichthe first direction corresponds to a past time point of the timeline andthe second direction corresponds to a future time point of the timeline.20. The mobile terminal of claim 18, wherein the timeline comprises apast time point, a present time point or a future time point.
 21. Themobile terminal of claim 18, wherein the controller is furtherconfigured to display on the display at least one indicator relative tothe user interface to indicate a specific time point at which a state ofthe object changes.
 22. The mobile terminal of claim 15, wherein thedisplayed information comprises weather information, event information,financial information or traffic information.